Multicriteria Evaluation in Computer Game-Playing, and its Relation to AI Planning

Games are a popular test bed for AI research. Many of the search techniques that are currently used in areas such as single-agent search and AI planning have been originally developed for games such as chess. Games share one fundamental problem with many other fields such as AI planning or operations research: how to evaluate and compare complex states? The classical approach is to ‘boil down’ state evaluation to a single scalar value. However, a single value is often not rich enough to allow meaningful comparisons between states, and to efficiently control a search. In the context of games research, a number of search methods using multicriteria evaluation have been developed in recent years. This paper surveys these approaches, and outlines a possible joint research agenda for the fields of AI planning and game-playing in the domain of multicriteria evaluation. Introduction The need for multicriteria evaluation techniques in gameplaying programs is not immediately obvious. All popular games have final outcomes that are scalar, be it windraw-loss as in chess, the number of points in games such as Go or Awari, or the amount of money in casino games. In games that are simple enough to allow a complete analysis, the exact value of a game position can be computed by the minimax evaluation rule (von Neumann 1928; von Neumann & Morgenstern 1947). However, complex games such as chess or Go rarely allow a complete analysis. Evaluation problems arise quickly when a player’s knowledge about a game is less than perfect. In place of an intractable complete analysis, games are usually analyzed by a deep but far from exhaustive search using a scalar-valued heuristic evaluation function. Another source of complexity are games where the complete game state is not known to a player because of hidden information such as the cards in other player’s hands in most card games, or because of chance events such as dice throws or cards drawn from a deck during a game. In this survey we will look at several techniques that use multicriteria evaluation in games. We start by summarizing some basic facts about the structures used in multicriteria evaluation: vector dominance, as used in multiobjective Copyright c 2002, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. evaluations, and general partially ordered sets. Vector dominance defines a specific kind of partial order, and in turn each finite-dimensional partial order can be represented by a vector with the same dominance relation that is used in multiobjective evaluation. The main part of the paper consists of an overview of two topics that the author has worked on: the search method of partial order bounding, and a class of games called combinatorial games which are based on a partial order of game values. We also briefly survey other related work on multicriteria techniques in games. The final, mostly speculative part of the paper discusses possible relations between the two fields of game playing and AI planning. How can multicriteria planning methods be used in game programs? And how can techniques developed for game tree search be used in multicriteria planning? Background In this section we describe the sum-of-features model for scalar evaluation, give definitions of multiobjective and partial order structures, and point out their close correspondence, at least in theory. The standard scalar approach: weighted sum of features In the standard model of computer game-playing, position evaluation is a two step process. The first step maps a game position to an abstract representation. A number of relevant attributes are computed and collected in a high-dimensional feature vector . Within such a vector, single features are usually of a simple type such as 0-1 (boolean), integer, or real. Given a feature vector and an integeror real-valued weight vector , a scalar-valued evaluation is computed as the weighted sum . The weighted sum approach to evaluation has been very successful in practice. It has proven to be a useful abstraction mechanism, with many desirable properties, such as simplicity, and ease of use in efficient minimax-based algorithms. Furthermore, in some games there is a natural mapping of positions to a numerical evaluation, for example the expected number of captured pieces in Awari or the balance of territory in Go. In games that end in a simpler outcome such as win, loss or draw, a scalar evaluation can be interpreted as a measure of the relative chance of winning. Despite the great success of the weighted sum approach to ev aluation, the method has quite a few weaknesses, and many of the alternative methods discussed in the survey (Junghanns 1998) were designed to address such weaknesses. The main drawback of using a single number for evaluation is that information is lost. All kinds of features are weighted, added and compared, even those for which addition and comparison do not really make sense. Problem topics include unstable positions, long term strategic features, and close-to-terminal positions. For a detailed discussion see (Müller 2001b). It is therefore natural to consider richer evaluation structures, such as partial orders. Partially Ordered Sets Our definitions follow standard conventions. For more information see textbooks such as (Stanley 1997; Trotter 1992). A partially ordered set or poset is a set (also called ) together with a reflexive, antisymmetric and transitive binary relation . The dual relation is defined by ! #"